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ABSTRACT 



This report addresses the relationship between the school 
accountability measures required for Title I schools and those put in place 
by state and local jurisdictions. The document's main focus is on the degree 
of coherence and compatibility among the multiple accountability systems, as 
well as the level and coordination of support provided to schools in need of 
improvement. The report analyzes two state-administered systems (Kentucky and 
Maryland) , two locally developed systems (San Francisco and Chicago) , and one 
large urban district responding to the state-based accountability 
requirements (New York City) . Five major findings regarding the 
identification and support of low-performing schools in these jurisdictions 
emerged; (1) most low-performing schools identified by state or local 
accountability systems are Title I schools; (2) considerable overlap exists 
between the schools identified for improvement under Title I and those 
identified in the jurisdictional systems; (3) where there is no overlap, the 
discrepancy stems primarily from differences in the criteria used for 
identification; (4) the lack of specified consequences for Title I schools 
that fail to improve contributes to a relatively low saliency for Title 
accountability in these jurisdictions; and (5) support provided to 
low-performing schools (identified by either Title I or jurisdictional 
accountability systems) varies substantially from jurisdiction to 
jurisdiction. (Author) 
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Foreword 



In 1996, the Office of Educational Research and Improvement (OERI) of the U.S. 
Department of Education (ED) funded a study of reconstitution as a remedy for school failure as 
part of the work of the Consortium for Policy Research in Education (CPRE). Five universities 
are members of the consortium, which was created on 1985 to improve elementary and 
secondary education through research on policy, finance, governance, and school reform. 
Members include Harvard and Stanford universities and the universities of Michigan, 
Pennsylvania, and Wisconsin-Madison. The consortium’s offices are located at the University of 
Pennsylvania’s Graduate School of Education. 

During the first year of the reconstitution study, CPRE co-sponsored a national meeting 
of state and local policy makers and educators who were actively engaged in designing or 
carrying out high stakes school accountability policies, including school reconstitution and 
probation. Similar meetings of state level educators had been convened or planned by other 
organizations, including the Education Commission of the States and the Council of Chief State 
School Officers. A pattern emerged in each of these meetings that caused some concern among 
researchers and ED officials involved: throughout the discussions, local and state policy makers 
and administrators made little if any mention of Title I of the Elementary and Secondary 
Education Act (ESEA). Criteria for school identification was not being linked to emerging 
definitions of adequate yearly progress nor was Title I mentioned as a source of support in the 
low-performing schools targeted for improvement. Given that Title I requires states and/or 
districts to hold schools receiving Title I funds accountable for improved student performance, 
and given the expectation that most of the lowest performing schools identified by the states and 
districts represented at these meetings were Title I schools, this lack of explicit attention to 
Title I generated the following questions for investigation: What is the relationship between the 
systemwide provisions for school accountability being developed by some states and districts 
and the provisions for accountability required and being implemented for Title I schools? 

To address this question, the Planning and Evaluation Service (PES) of the ED decided to 
extend the scope and funding for the CPRE study on reconstitution. The purpose of this 



extension was to focus on the overlap and coordination of the state and local accountability 
systems with the systems being established under the auspices of Title I, to look into supports 
offered to schools identified by such systems, and to make recommendations for strengthening 
accountability and support systems for low-performing Title I schools. Originally, four 
jurisdictions — two states (Kentucky and Maryland) and two districts (Chicago and San 
Francisco) - were targeted for study. Each of these had established a jurisdiction-wide, high 
stakes accountability system for its low-performing schools. A fifth jurisdiction - New York 
City- was added because of the relatively long history of the Schools Under Registration Review 
(SURR) program in that state. 

This report presents a brief review of the findings and recommendations of the 
investigation of Title I accountability in relation to the state and local accountability systems in 
these five jurisdictions. The results of the parent study of school reconstitution, on which some 
of the recommendations are based, will be available in the fall of 1999. 




VI 



Executive Summary 



Background (Section I) 

During the past decade, the problem of persistently low-performing schools has received 
increasing attention from educators, policy makers, and the general public. In response, more 
than 22 states and several major districts have instituted policies requiring school-level 
accountability for student performance and sanctions for schools that fail to improve. In 1994, 
the federal government joined this trend by strengthening the accountability provisions in the 
Elementary and Secondary Education Act (ESEA) and imposing sanctions for Title I schools that 
fail to demonstrate adequate progress on standards-based measures of student performance. 



This report addresses the relationship between the school accountability measures 
required for Title I schools and those put in place by state and local jurisdictions for all their 
schools. Our main focus was on the degree of coherence and compatibility among the multiple 
accountability systems, as well as the level and coordination of support provided to schools in 
need of improvement. The sample selected for study included two state-administered systems 
(Kentucky and Maryland), two locally developed systems (San Francisco and Chicago), and one 
large urban district responding to the state-based accountability requirements (New York City). 
Each of these jurisdictions had developed its system-wide accountability policies independent of, 
and generally prior to, the new Title I provisions. 

Findings (Section II) 



Five major findings regarding the identification and support of low performing schools in 
these jurisdictions emerged. 



• The majority of the low-performing schools identified by the state or local 
accountability systems are Title I schools. 

• Considerable overlap exists between the schools identified for improvement under 
Title I and those identified in the jurisdictional systems. This overlap is not absolute, 
however, and varies considerably among the jurisdictions. 



• Where identification of schools by the jurisdictional and Title I systems does not 
overlap, the discrepancy stems primarily from differences in the criteria used for 
identification. These differences in criteria, in turn, seem to derive from several inter- 
related factors: differences in the purposes of the two systems, differences in the 
official who (i.e, which unit or level) identifies the schools, and general problems in 
implementation and administrative coordination. 

• Lack of specified consequences for Title I schools that fail to improve seems to 
contribute to a relatively low saliency for Title I accountability in these jurisdictions. 

• Support provided to low-performing schools (identified by either Title I or 
jurisdictional accountability systems) varies substantially from jurisdiction to 
jurisdiction and often from school to school. 

Conclusions (Section III) 

Based on these findings, we drew several conclusions pertinent to Title I accountability. 

Compliance: All of the jurisdictions were attempting to comply with the requirements of 
Title I accountability and were doing so primarily by trying to fold Title I accountability into 
jurisdiction-wide systems in some way. Strategies ranged from complete coherence of the 
systems (Kentucky), to negotiations for state Title I acceptance of the jurisdictional system (San 
Francisco), to a division of labor between the systems (Chicago). 

Coherence : Despite attempts to make the systems coherent. Title I schools often appear 
to be subject to dual systems of accountability. While not an immediate problem in most 
jurisdictions, we expect the lack of congruence among the systems to increase in saliency as the 
more severe sanctions for Title I accountability begin to carried out. 

The Urban Problem : The problem of low performance, accountability, and assistance for 
schools is particularly acute in urban areas. Schools in large urban districts are more likely to be 
identified by state accountability systems, and the problem of low performance is likely to be 
more pervasive in these districts. Special provisions and attention will be required to address this 
problem. 
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Recommendations (Section III) 



Although not a specific charge of this study, several recommendations for reauthorization 
of ESEA emerged from this study. Briefly, these are: 



1 . In reauthorizing ESEA, Congress should remove or revise those processes or 
provisions that foster the development of a dual system of school accountability for 
Title I schools. 

2. The reauthorized ESEA should place greater emphasis on ensuring that schools in 
need of improvement receive appropriate, consistent, and intensive assistance from 
districts and states. 

3. Several aspects of Title I accountability guidance regarding criteria and consequences 
should be strengthened, either by incorporating them directly into law or by working 
closely with states to foster their implementation. These include the use of multiple 
criteria for accountability decisions, the incorporation of disaggregated student 
assessment results into accountability criteria, and the implementation of corrective 
actions appropriate for the specific problems in identified schools. 
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Section I. Context and Purposes 



"We cannot and must not tolerate failing schools. To meet the new expectations 
of the American people, we must confront the alTtoo-common problem of schools 
that are low achieving and even dangerous ...If a school is bad and can 't be 
changed, reconstitute it or close it down. ” 

Richard W. Riley, Feb. 18, 1997 

In the mid 1990s, after over a decade of education reform, the public and its policy- 
makers turned increasing attention to the problem of persistently failing schools - schools that, 
even after multiple reform efforts, remained at the bottom on measures of student performance. 
Policy talk in states and districts began to focus on the need for greater accountability and more 
drastic interventions in these schools. Meanwhile, in educational and mass media, occasional 
reports of such interventions, like the 1993 reconstitution of Rusk Elementary School in 
Houston, began to appear. 

As is often the case with education reform trends, the move from policy talk and isolated 
action to full-blown policy systems occurred at break-neck speed. San Francisco, which had 
reconstituted four schools as part of a desegregation consent decree in 1983, began to do so on an 
annual basis in 1994, preceding such action with a two-stage process of identification and 
assistance. The following year, the Illinois legislature approved a similar multi-stage process for 
Chicago, which in turn shocked local and national educators by placing 109 of its 554 schools on 
probation in 1996, threatening them with reconstitution if they did not show improvement. 

Nor were state governments standing idly by while districts took up the school 
accountability banner. By 1997, 22 states had passed “academic bankruptcy laws” allowing 
them to intervene directly in low-performing districts and schools. Kentucky was among the 
first such states, having written a system of school rewards and sanctions for student 
performance into their Kentucky Education Reform Act (KERA) as early as 1990; the first 



‘ Olson, Lynn (1995). Starting Over. Teacher Magazine, 6. 38-43. 



groups of schools were identified under those provisions in 1994. That same year, Maryland 
started placing schools on an annual list of reconstitution-eligible sites, while New York, Texas, 
and other states developed their own versions of similar procedures. 

Through these actions, states and districts were setting in motion interventions and 
policies to attend to the problem of persistently low-performing schools. By and large, those 
policies focus on holding schools accountable for the performance of their students and 
instituting consequences for continued failure to improve. In light of this trend, it was not 
surprising when the federal government also upped the ante on school accountability. It did so 
by instituting a key set of provisions in the 1994 reauthorization of the Elementary and 
Secondary Education Act requiring states to hold Title I schools and their districts accountable 
for their students’ progress toward meeting the standards applicable for all students. While the 
notion that schools receiving Title I funds should have to show progress in student performance 
first appeared in the Hawkins-Stafford Amendments to ESEA in 1988; the 1994 reauthorization, 
with its emphasis on adequate yearly progress and corrective actions for non-improving schools, 
was an expansion and deepening of the earlier provisions. 

Study Purpose 

Current attention to school-based accountability and the institution of high stakes 
measures for low-performing schools thus derives from several sources - state legislation and 
policies; local education agencies; and federal programs, particularly Title I. An obvious 
question is to what extent these various systems for school accountability are working together to 
improve achievement for all students. That question was the motivation behind this study. 

The study reported here focused on five jurisdictions that in the previous few years had 
instituted high stakes school accountability systems for the schools under their authority - 
Kentucky, Maryland, San Francisco, Chicago, and New York. Our goal was to see how these 
jurisdictions were incorporating the Title I provisions into their own systems. 
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• Were the Title I systems and the local and state systems of school accountability 
coherent and compatible? 

• Was Title I being viewed and used as a resource for assisting low-performing schools 
- or at least low performing Title I schools - identified through the local and state 
systems? 

• And finally, what were some of the lessons we were learning about high stakes 
accountability systems that might be used to improve the accountability measures in 
Title I, particularly in preparation for reauthorization of ESEA in 1999? 

Because policy coherence is a central goal of standards-based reform, we paid particular 
attention to the degree of congruence between Title I and the jurisdictional system. 

Data: This study was part of a larger investigation of school accountability policies and 
practices in these and other jurisdictions. Data collection for the larger study included document 
review (policies, media and research reports, etc.), interviews and meetings with state and local 
policy makers and administrators, site visits and case studies of reconstituted schools. 

Additional interviews, document review and analysis of school-level data were conducted for 
this study in the five jurisdictions. 

Report organization : To address the questions above requires an understanding of the 
key provisions in Title I and in the jurisdictions in question. These provisions are outlined in the 
remainder of Section I. Section II then reports our findings on the congruence between the 
systems created by Title I and those established by the jurisdictions for all their schools. Section 
III discusses the findings and advances several recommendations for improving accountability 
provisions in the next reauthorization of lASA. 

Title I Accountability and lASA 

Title I accountability can only be understood within the context of the overall framework 
of the 1994 reauthorized ESEA. Four aspects of that framework have particular significance for 
the purposes of this study. They are: the centrality of standards, the emphasis on policy 
coherence, the attention to professional development and capacity, and performance-based 
accountability. 

1 
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Standards 



Standards are the lynchpin of the 1994 reauthorization and represent a major departure 
from previous practice in Title I/Chapter 1. The underlying notion is that all children should 
have the opportunity to achieve to the same high standards. This approach runs counter to the 
conventional dual system with challenging curriculum and expectations for some students and a 
watered-down remedial basic skills approach for others. Too often in the past, Title I/Chapter 1 
contributed to this dual system by offering fragmented remedial instruction in pull-out sessions 
taught by ill-prepared personnel^. With the 1994 reauthorization, services to Title I students and 
schools are intended to enable the children served by Title I to meet the challenging standards 
established by the state for all its children. Moreover, the legislation requires the states to 
institute such standards (content and student performance standards) in at least reading and math 
by the 1997-98 school year; final state assessments aligned with those standards are to be in 
place by 2000-01. 

Coherence 

If standards are the lynchpin, coherence is the goal - coherence among the programs of 
the Act, coherence among the levels of governance (federal, state, local), coherence of policy 
components (curriculum, assessments, professional development and accountability), and 
coherence in the educational program available for all students. Much of the criticism of 
Chapter 1 under previous versions of ESEA was the fragmentation that it engendered, both in the 
actual program for students - with the emphasis on pull-outs and isolated skills - and also the 
fragmentation in the system as a whole— with all the separate accounting mechanisms and 
personnel and the stove-pipe administration of services for Title I students. The reauthorized 
ESEA was designed to address that fragmentation and to promote coherence, primarily by 
helping to pull together all aspects of the instructional program and institutional organization to 
foster achievement of the state standards. 



^ According to the National Assessment for Chapter 1 . 
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Several specific aspects of Title I reflect this emphasis on coherence. One is the 
expansion of schoolwide programs and the move away from pull-out services for individual 
students. Tied to this is the promotion of consolidated planning, in which Title I and other 
federal monies are to be seen as assistance for a school in developing its overall program to meet 
the needs of its students and to foster their achievement of the standards. The comprehensive 
assistance centers, which pull together assistance for all the various titles in ESEA, are another 
indication of this effort. And finally, the flexibility that states have in developing measures and 
systems and programs that are consistent with their own standards and the thrust of their 
educational program is yet another. The availability of this flexibility is particularly evident in 
the Ed-Flex states, which are able to waive a number of requirements in federal statutes to 
enhance the coherence and effectiveness of their overall efforts. 

Professional development and capacity building 

A third integral aspect of ESEA is the importance placed on notions of capacity-building 
and professional development in support for schools and teachers. A key criticism of earlier 
versions of Chapter 1 was that often the teachers responsible for instruction - or m many cases 
the aides responsible for instruction - to Chapter 1 students were ill-prepared and unable to 
provide the kind and quality of instruction that students needed. For this reason, an emphasis in 
recommendations prior to the 1994 reauthorization and in the reauthorization itself was on the 
need for professional development, encouraging schools and districts to use Title I funds for 
professional learning - not only that of Title I teachers but of all teachers in the school, as any of 
them may serve Title I students. The establishment of more comprehensive technical support, 
and mechanisms like the school support teams to assist schools in planning and developing their 
schoolwide programs is another example of this emphasis. In addition, as will be noted, 
provisions were included for providing additional assistance to Title I schools that were 
experiencing difficulties. Indeed, schools identified as in need of improvement are required to 
allocate at least ten percent of their Title I funds over a two year period to professional 
development. 
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Performance accountability 



It is the accountability provisions of Title I that are the most directly central to this study. 
Unlike earlier iterations of Chapter 1 /Title I, in which the thrust of the accountability provisions 
focused on accounting for the use of Title I dollars, the thrust of accountability in Title I since 
1 994 is on student performance. Thus, states are required to hold districts (LEAs) and schools 
receiving Title I funds accountable for ensuring that all students meet the state’s content and 
performance standards. The primary mechanism for doing so is the state’s definition of adequate 
yearly progress (AYP), which is based on a model of continuous improvement toward the 
standards. According to federal guidance for Title I, AYP “must be defined in a manner that - 

• Results in continuous and substantial, yearly improvement of each school and LEA 
sufficient to achieve the goal of all children served under Title I, particularly 
economically disadvantaged and limited-English proficient children, meeting the 
State’s proficient and advanced levels of performance; 

• Is sufficiently rigorous to achieve that goal within an appropriate time frame; and 

• Links progress primarily to performance on the State’s final assessment while 
permitting progress to be established in part through the use of other measures, such 
as dropout, retention, and attendance rates. 

The specific definitions of AYP in use by the five jurisdictions in this study are 
summarized in text form in Appendix A of this report and in tabular form in Appendix B. 



^ Taken from the Title I Guidance on Standards and Accountability section III provided by the U.S. Department of 
Education 
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Another central feature of Title I accountability is the designation of schools and districts 

as “in need of 
improvement” when they 
fail to achieve AYP for two 
consecutive years. Such 
designation brings with it 
several requirements: the 
development of an 
improvement plan, the 
allocation of funds for 
professional development, 
provision of technical 
assistance, and — if failure 
to improve continues — some form of corrective intervention. “Corrective actions are intended 
to take place in the third year following identification. It should be noted, however, that the 
inclusion of corrective actions is new to this reauthorization and the legislation precludes states 
from taking the more severe actions listed in this legislation— withholding funds, revoking 
schoolwide program authority and decision-making structures, reconstitution, or authorizing 
student transfers — until their final assessments and final definitions of AYP are in place; that is, 
no later than 2000-2001 . Again, the specific features of the school improvement provisions in 
the five jurisdictions are summarized in Appendices A and B. 



Timeline for Title I Accountability 



flgas DE03 lEp" 



Transition Period: 

Identify schools and districts in need 
of improvement, using accurate 
information about academic progress 




Final Period: 

Adequate Yearly Progress 
definitions to be based on 
Final Assessments aligned 
with State standards 



High Stakes School Accountability: The Jurisdictional Systems 

To examine the degree to which Title I is incorporated into the systems in these five 
jurisdictions, it is necessary first to understand both the general framework of accountability in 
the jurisdictions and some of the particulars of the policies. Our larger study of reconstitution 
and other high stakes accountability systems suggests the outlines of that general framework. 
While noting that policies vary in important respects from jurisdiction to jurisdiction across the 
country, we find them to reflect some common assumptions about accountability and 
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improvement efforts in schools. Taken together, these assumptions form a generic “theory of 
action” (Argyris and Schon, 1978; Weiss, 1995) common to all jurisdictions in this study. 



The elements of this theory of action are as follows: 



• The accountability system should be aligned with educational goals. If increased 
student achievement is the goal, then student achievement is what educators should 
be accountable for and what the system should monitor. The belief is that the 
accountability system will draw attention to the desired goals and focus effort on their 
achievement. 

• The school should be the basic unit of accountability. Two beliefs support use of the 
schools as the unit of accountability. One is that student performance is cumulative 
and influenced by the entire school program and climate. A second is that a system 
of collective (school-based) accountability will encourage educators at the school to 
work together, thus increasing overall capacity and performance. 

• The threat of negative sanctions will motivate educators to work harder, thus 
improving student achievement. The underlying belief here is that persistently low 
student performance stems primarily from educators’ lack of will. Whether 
policymakers attribute this lack of will to institutional racism or to low expectations, 
to poor school leadership or to well-meaning teachers becoming discouraged by 
external conditions, the belief is that educators need some external push to do what is 
necessary to raise student achievement. Negative sanctions are to provide that push. 

• Accountability measures should be accompanied with capacity building. Policy 
makers recognize that limited capacity (knowledge, skills, and resources) at the 
school also contributes to low performance. School probation policies, therefore, also 
provide assistance and professional developmenfopportunities to help build needed 
capacity in the schools. 

• Goal-setting, planning, and monitoring results are critical to improvement. The 
underlying belief is that very broadly defined goals do little to focus attention, suggest 
strategies, or provide feedback for improvement. Schools need specific, measurable 
goals and well-specified plails for achieving them or for modifying practice if they 
are not achieved. All high-stakes school accountability systems currently in use, 
therefore, have specified student achievement targets and include site based planning 
mechanisms for achieving those targets. 

• Some schools are simply beyond help and should be closed down, reconstituted, or 
taken over. All the accountability systems discussed here incorporate some last resort 
measure to intervene and protect students wh^h a school continues to fail after 
previous interventions. In Chicago and San Francisco, this last step involves the 
removal and replacement of school administrators and staff as well as some additional 
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measures. In New York and Maryland, the last resort is mainly a governance change 
- removing decision-making authority from the school and district (state takeover) - 
or closure. 

Reflecting these common assumptions, the policies in these five jurisdictions nonetheless 
vary considerably in their particulars. We summarize those particulars in the remainder of this 
section. It should be noted that we chose these five jurisdictions in order to see both local and 
state interaction with Title I. Two of the systems examined, Chicago and San Francisco, are 
locally developed and implemented. Both are relatively large urban districts with very diverse 
student populations. Two of the others, Kentucky and Maryland, are statewide systems of 
accountability in which the state holds individual schools accountable for performance, with 
differing involvement of LEAs in the process. The final jurisdiction. New York City, falls 
between these two approaches in that it is a large urban district operating within a statewide 
accountability system'*. 

The remainder of this section describes the school-based accountability systems operating 
in each of the five jurisdictions. Each description includes the designations of improvement and 
the sanctions used in the jurisdictional system, the criteria employed for placing schools in those 
designations, the support available to schools once they’ve been identified, and the numbers of 
schools in the categories in 1997-98, the year in which data for this study were collected. 

Kentucky 

School-based accountability for student performance has been a central component of the 
Kentucky system since passage of the Kentucky Education Reform Act (KERA) in 1990. What 
is unique about the Kentucky system is that for the past nine years schools have been judged 
solely on whether their students were making progress (relative to the school’s previous 
performance) toward the achievement standards established by the state. The model of 



Note Illinois and California have or are developing statewide systems of school accountability which will interact 
with the locally designed systems in Chicago and San Francisco. The focal systems for this study, however, were 
those devised and implemented by the local district. Interactions with state systems will be noted as appropriate. 
New York City by contrast developed specific measures for responding to the designation of its schools as SURR 
schools. Thus the primary accountability system for New York City was that of the state. 
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continuous improvement underlying this system is highly compatible with the concept of 
adequate yearly progress outlined in the Title I legislation. 

The instrument used forjudging school performance in Kentucky is the state assessment 
system, which until spring 1998 was KIRIS (Kentucky Instructional Results Information 
System). (A new assessment — the Kentucky Achievement Test — is currently under 
development.) The system measures the progress of each school in the state, relative to the 
school’s earlier performance on KIRIS and to specified biennial targets. Those targets are based 
on two assumptions: 1) that all students in a school should be scoring at the “proficient” level; 
and 2) that schools should be able to accomplish this goal within a twenty year period. To 
achieve this ultimate goal the state established a series of two-year performance targets for each 
cycle. These performance targets, or “improvement goals,” are calculated from a baseline score, 
defined as the school’s accountability index at the beginning of the two-year cycle. The 
Improvement Goal is set at 10% of the difference between the baseline and 100. Thus, the 
improvement goal for a school with a base line accountability index of 50 would be 55 for that 
two-year cycle. 

Obviously, under this system, low-performing schools have been expected to make 
greater proportional gains in performance, as they were the farthest from the goal at the outset of 
the program. Nevertheless, ALL schools in Kentucky have been subject to accountability for 
their individually specified targets. Those surpassing their improvement targets have been 
eligible for rewards, regardless of their absolute performance level. Similarly, both “high 
performing” and “low-performing” schools have been subject to sanctions if they failed to make 
the specified gains. Schools whose performance index falls relative to their baseline at the 
beginning of the biennium, but whose decline is less than five percentage points, are designated 
as “schools in decline.” “Schools in crisis” are those whose performance falls by more than five 
points or who are in decline for two cycles. Schools are re-designated every two years, based on 
their meeting or not meeting the improvement targets for that biennium. Our focus for the 
purposes of this study was schools in “decline” or “crisis.” 
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Through spring 1998, assistance to all schools designated in decline or crisis was 
provided in the form of a “Distinguished Educator,” who worked with the school over a two-year 
period. Distinguished-Educators trained the staff in how to use data for instructional decisions, 
assisted in the school planning process, and helped to align curriculum and instruction with state 
standards. Distinguished Educators were highly effective teachers and administrators selected 
and trained by the state and released from their regular duties for this two-year period. In 
schools designated as “in crisis,” Distinguished Educators also had the authority to evaluate and 
replace school staff This system of support, like the state assessment, is currently being revised, 
based on new state legislation in 1998. 

At the time of this study (1997-98), 302 schools for the previous cycle had failed to meet 
their improvement target. Eighty-two of these were designated in decline, and nine were in 
crisis. The total of 91 decline and crisis schools were located throughout the state and 
demonstrated varying levels of absolute aggregate performance. 

Maryland 

Maryland’s high stakes accountability system for low-performing schools was instituted 
under the leadership and authority of state superintendent Nancy Grasmick in 1994 but traces its 
origins to a 1989 gubernatorial commission that laid out a similar performance-based system. 
Beginning with a belief that children have a right not to be forced to attend a failing school by 
accident of where they live, state policy allows the state Board to directly intervene in failed 
schools by removing governance authority from the local district. This state “take-over” of 
individual schools is called reconstitution. To date, no schools in Maryland have been 
reconstituted in this manner. Instead, the Maryland leadership have focused on declaring schools 
“Reconstitution Eligible” and requiring local school systems to develop improvement plans for 
those schools. A larger “alert list” of schools provides LEAs with advance information about 
schools in need of assistance. 

The criteria for identifying Reconstitution Eligible schools is the state performance index 
(SPI) which is composed of aggregate assessment scores on Maryland State Performance 
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Assessment Program (MSPAP) or the Maryland Functional Tests (high school), attendance rates, 
and drop out rates (high school). The state has established a goal of 70% of students at proficient 
levels of MSPAP and it targets for intervention those schools that are well below that standard 
and declining in performance. The number of schools targeted each year is based in part on the 
perceived capacity of the SEA to provide assistance. 

The assistance provided, however, is mainly the responsibility of the local districts, each 
of which must also submit a plan for improvement^. Specific interventions for Reconstitution 
Eligible schools vary and may include replacement of administrators and staff (zero-based 
staffing) as well as restructuring and professional development. 

When the system was instituted in 1994, two Baltimore City high schools were 
designated as Reconstitution Eligible. Since that time, 90 schools have been named 
Reconstitution Eligible, 79 of them in Baltimore City. The concentration of Reconstitution 
Eligible schools in Baltimore, in conjunction with other legal and financial matters, contributed 
to a legal settlement between the city and the state, known as the Baltimore Partnership. The 
Partnership, which came into existence in 1997, infuses additional funding into city schools and 
revamps the governance structure of the school system, replacing the old Board of 
Commissioners appointed by the mayor with a new board jointly appointed by the mayor and the 
governor. At the time of this study Maryland had no established criteria for removing schools 
from the Reconstitution Eligible designation. 

Chicago 

Prior to the 1995 revision of the Illinois School Code,^ the central office of Chicago 
Public Schools lacked the authority to intervene in low-performing schools. The education 
reform legislation passed by the Illinois legislature in 1988 had decentralized that authority to 

^ A policy slated for implementation in July 1999 will provide greater guidance and assistance to schools that have 
been in Program Improvement for three consecutive years. These schools will be required to develop and 
implement a program that incorporates the nine points of a researched based design introduced by the 
Comprehensive School Reform Demonstration Program. Schools will be provided assistance from the state Title I 
office and can solicit assistance from external agents. 

^ 1995 Amendatory Act to the Illinois School Code; Public Act 89-15. 
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Local School Councils. Local School Councils are governing bodies elected by the school 
community at each site and vested with the authority to hire the principal and set the instructional 
and management policies for the school. Reformers believed that such decentralization would 
create a system that was more responsive to student needs and would lead to higher achievement. 
For many Chicago schools, this governance change worked well. Others, however, remained 
mired in failure. Faced with a large number of persistently failing schools, the Illinois legislature 
revised the statutes, establishing a Chief Executive Officer for Chicago Public Schools, 
appointed by the mayor and vested with the authority to intervene in low-performing schools. 

The resulting system of school accountability was established under this legislation and the 
leadership of the current CEO, Paul Valias. 

Chicago’s accountability system involves three stages: remediation, probation, and 
reconstitution. Reflecting the emphasis in Chicago, our focus for this study was on the probation 
and reconstitution categories. CPS places schools on probation if fewer than 15% of their 
students are reading at grade level on the ITBS and removes them from probation when 20% of 
their students are reading at grade level. Reconstitution can occur if a school on probation fails 
to improve performance. Actual reconstitution decisions are made case by case, based on an 
evaluation of progress and potential for progress in the school. If the school is reconstituted, the 
school administration and staff are removed from their positions. Given an opportunity to 
reapply for their jobs in that school, the staff may be either rehired or reassigned and replaced. 

CPS has established a fairly elaborate and multi-faceted system of support and 
monitoring for probation and reconstituted schools. Each school receives the services of a 
probation manager, who assists the leadership in developing and monitoring the implementation 
of the school improvement plan. In addition, each school contracts (with financial assistance 
from CPS) with an external partner for on-going professional development and capacity building 
services. The external partners are mostly university-based groups with varying approaches to 
school improvement. Reliance on external support providers derives, in part, from a CPS desire 
to involve local universities in improvement efforts. CPS administrators also believed, on the 
one hand, that support would be better received from outside the system, and, on the other, that 
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CPS central office simply did not have the capacity to meet the needs of all the schools requiring 
assistance. 

In 1996, 109 (approximately one-fifth) of Chicago public schools were placed on 
probation. In 1997-98 several schools were removed from the list and others put on, so that 
during the academic year 115 schools were on probation, while seven high schools were 
reconstituted^. 

Responsibility for holding Chicago schools accountable for performance does not rest 
solely with the district administration, however. Chicago also exists within a larger state system 
of accountability, which differs in important ways from the local. Like Chicago’s system, the 
state has three levels of designation for low-performing schools, but the criteria for identification 
in the state system is the Illinois Goals Achievement Program (IGAP), rather than the ITBS, and 
specific consequences for continued failure to improve have not been spelled out. The state 
requirements are summarized in Appendices A and B and will be discussed in Section II as tied 
to definitions of adequate yearly progress. 

San Francisco 

San Francisco’s system of school accountability derives from a 1983 desegregation 
consent decree and the 1993 amendment to that decree. In 1984, four San Francisco schools 
were reconstituted” in an effort to desegregate schools in the Bay View Hunter’s Point section 
of the city and to improve the education available to the city’s African American and Latino 
students. Based on advice from an expert panel convened by the court to review the first ten 
years of the decree, the parties amended the agreement to include a call for “annually 
reconstituting at least three schools until the task is completed.” 

To implement the amended decree, the district, under the leadership of the new 
superintendent Wildemar Rojas, introduced a two-stage process in which low-performing 
schools were first identified for the Comprehensive School Improvement Program (CSIP) and 



’ Currently 91 schools are on probation and no new schools have been reconstituted. 



given a year to improve. At the end of the academic year, CSIP schools could be continued in 
the program, “graduated,” or reconstituted. 

The decision to place schools in CSIP was based on a 100-point index comprised of 17 
indicators of need and performance. Student achievement, measured by the CTBS, played 
heavily in the identification process. The decision to reconstitute a school after it had been in 
CSIP was based on a set of evenly weighted quantitative and qualitative indicators, including a 
visit to the school by the superintendent and his district team. In San Francisco, when a school is 
reconstituted, the entire staff is “vacated” and the new staff must agree to adhere to a set of 
philosophical tenets designed to ensure high expectations for all students. Emphasis in 
reconstituted schools is also placed on the use of technology, flexible class sizes, professional 
development and parental involvement to improve instruction and achievement. 

Schools targeted by the consent decree are allocated additional discretionary monies as 
part of the desegregation efforts. In addition, some additional monies are available to CSIP and 
reconstituted schools, whether or not they are among those targeted by the decree. Starting in 
^ 997 _ 9 g^ these monies were held centrally and are allocated to the schools for specific purposes, 
primarily professional development activities. Schools have also had the opportunity to receive 
external consulting, and in the 1997-98 academic year, the district instituted an interdepartmental 
team to coordinate services and assistance to CSIP and reconstituted schools. 

Since 1994, 24 schools have been placed into CSIP; 14 of these have now graduated from 
the list, 10 schools have been reconstituted. In May 1997, district Superintendent Rojas signed a 
tentative agreement with the president of the local teachers union to develop alternatives to 
reconstitution*. Two schools have been reconstituted since that initial agreement was signed; 
none have been placed in CSIP. A new three-tiered accountability system has been planned for 
the 1998-99 school year, but to date no schools have been placed in that system. 



* The Trust Agreement remained as of March 1999. 
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New York City and SURR 



New York’s state system of school accountability, Schools Under Registration Review 
(SURR), was initiated in 1989, though a similar program, the Comprehensive Assessment Report 
(CAR)^ had begun in 1985. The SURR program is predicated on a state requirement that all 
schools must be registered to operate in New York and on the authority vested in the state 
department of education to revoke the registration of schools not meeting state standards. 
Currently, schools placed in the SURR program have three years in which to meet state standards 
or be de-certified (i.e., closed down or “redesigned” with a new school plan). (Note: This time 
limit was instituted by State Commissioner Rick Mills in 1996-97; schools identified before that 
time were given only two years after this provision went into effect.) 

Schools are identified for SURR primarily through test scores on state assessments in 
reading and math in grades 3, 6, 8 (reading only), and 11. The state has established minimum 
performance standards on these assessments and requires 90% test-takers in each school to score 
at or above those standards. In addition, the schools’ annual drop out rate must be below 5%. 

As of 1996, the schools identified for SURR are those determined to be “farthest from the state 
performance standards.” Additionally, schools that offer their students “a poor learning 
environment” can also be identified for SURR. Three elements contribute to the state’s 
definition of a poor learning environment: poor performance on state standardized tests; parental 
complaints to the state; conditions such as violence, excessive absenteeism and suspension; or a 
high percentage of uncertified teachers. 

Once identified for SURR, schools are to receive both state and district assistance in their 
improvement efforts. All schools are required to develop a Comprehensive Education Plan 
(CEP). In an effort to promote district responsibility and accountability for improvement in 
SURR schools, the state has also required districts to sign off on the school’s CEP and, since 
1996, to submit a district Comprehensive Assistance Plan (CAP) for each SURR school in its 
jurisdiction. Support for SURR schools varies substantially and has been characterized as a 



’ Ascher, C., Ikeda, K., and Fruchter, N. School on Notice: A policy study of New York State 's 1996-97 Schools 
Under Registration Review Process, Institute for Education and Social Policy, New York University, 1998. 
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“patchwork” consisting of a state liaison, district assistance, and opportunities such as the AFT’s 
Teacher Centers located in many New York City schools or participation in various summer 
institutes and initiatives'®. In New York City, twelve of the lowest performing SURR schools 
receive concentrated assistance and monitoring through the auspices of the Chancellor’s District. 
Governance authority for these schools was temporarily removed from the Community School 
Districts by NYC Chancellor Rudy Crew under the authority granted to him by the state 
legislature in 1996. The Chancellor’s District has its own supervising superintendent whose goal 
has been to get the schools off the SURR list and to develop their capacity to stay off and 
continue improving. 

Between the academic years 1989-90 and 1996-97, 139 schools had been identified for 
SURR in New York state, with the vast majority of these being located in New York City. In the 
last year of that period, 99 schools were still in SURR, of which 92 were in NYC. The 1997-98 
school year saw a similar pattern, with 98 schools throughout the state identified for SURR, 94 
of these in NYC. Of these 94, 28 had been redesigned and were still designated SURR, and one 
was in the process of closure. Meanwhile, 18 previously identified schools were removed from 
the SURR list and 20 new schools added. 



See Ascher et al. 1998 




17 



Section II. Title I Accountability in the Context of State and Local 

Accountability Systems 

This section presents the key findings of our investigation into the relationship between 
Title I accountability and local/state accountability systems in these five jurisdictions. Our 
findings fall into three general categories: identification of low-performing schools; assistance 
provided to identified schools; and consequences of continued low performance. For each of 
these, our emphasis is on the issue of coherence and coordination - i.e., to what extent do state 
and local systems work with Title I to produce a coherent approach to school accountability and 
improvement? The section concludes with several overarching observations about the degree of 
coherence and coordination among the systems and presents recommendations for the ESEA 
reauthorization in 1999. 

Identifying Low-Performing Schools 

The criteria and processes for identifying and categorizing levels of performance are 
central features of any performance-based accountability system. Criteria serve to define not 
only the goals and targets of performance, but success and failure as well. They communicate to 
schools and the public the system’s expectations and provide an indication of who is to be held 
accountable. As discussed in Section I, the primary unit of accountability in each of the systems 
in this study is the school, though to varying degrees districts also enter in as accountable units. 

We were confronted with two main tasks in this analysis. One was to examine the 
congruence among the policies governing Title I accountability and those governing 
accountability for all schools in the given jurisdiction. The second task was to look at the degree 
of congruence in the results of those policies - that is, in which schools actually get identified. 
For the purposes of this report, we begin with the second of these tasks. We then consider the 
criteria and processes outlined in the policy as one set of explanatory variables for the 
congruence or incongruence in the schools identified. 

To address the issue of which schools get identified, we first obtained lists of schools in 
each relevant category. For example, in Maryland, we obtained from the state a list of all 
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schools designated as Reconstitution Eligible through spring 1998. We included the schools’ 
Title I status, grade range, location, and so forth. We then obtained the names and locations of 
Xitle I schools and Title I schools identified as in need of improvement. Since there is no 
consolidated list of such schools for the state, this required compiling the data from each 
county’s Title I report. We followed a similar procedure for each of the jurisdictions. From 
these data, we constructed a data set for each jurisdiction that included Title I schools, schools 
identified by the state and/or local accountability systems, and schools designated as “program 
improvement” or “schools in need of improvement” by Title I. Through this process, we were 
able to produce, we believe, a fairly accurate picture of the incidence and overlap of schools in 
each relevant category. 

Two observations about these data are worth noting, however. First, their collection was 
not what one might call straightforward. Title I lists and state/local lists were not held in 
common in any jurisdiction except Kentucky. Differing offices kept records of each group of 
schools. Moreover, routes to individuals who could actually supply the information was often 
circuitous — particularly with respect to Title I schools — and these individuals were often unable 
to provide information on policy processes or support systems. This pattern raises concerns 
about the degree of administrative coordination of systems, apart from or in addition to concerns 
about the particular schools identified. Second, because information on the schools had to be 
obtained from several sources, our conclusions may not jibe exactly with the assumed 
relationship among the systems in any given jurisdiction. 

Below, we present our findings from this analysis. We use Venn diagrams to graphically 
(though approximately) display the patterns we found. We address three central questions in this 
discussion: 

1 . What proportion of schools identified in the state and local systems are Title I 
schools? 

2. What is the degree of overlap between schools identified by the Title I criteria and 
those identified by the state and local accountability systems? 

3. How might we explain the degree of congruence or non-congruence in the two 
systems? 
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Finding One: The majority of the low-performing schools identified by the state or 
local accountability systems are Title I schools. (And, with the exception of Kentucky, the 
majority of identified Title I schools were higher poverty schools implementing schoolwide 
programs.) 

The proportion of schools designated as low-perfonning (on probation, Reconstitution 
Eligible, in decline, etc.) that are receiving Title I funding is displayed graphically in 
Figures la - le and in numerical form in Table 1. The proportion of Title I schools ranges from 
59% in Kentucky to 93% in Maryland. 

Two observations are worth noting. The first is that the high proportion of Title I schools 
identified by the state and local systems supports the concern that instigated this study. That is, 
large numbers of Title I schools are apparently subject to at least two systems of accountability 
for student performance — that required by the Title I provisions of ESEA and that of the state or 
local jurisdiction in which the school is located— and those systems may or may not be sending 
coherent messages to schools about the goals and targets of achievement. Given the large 
numbers of Title I schools involved, it thus makes sense to look further into the congruence and 
coordination among the systems. 
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Figure 1b. 
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Local Accountability and Title I Schools 
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Figure 1c. 
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A second observation is that while the proportion of Title I schools is substantial in all 
jurisdictions, it varies significantly from jurisdiction to jurisdiction. Not surprisingly, Kentucky 
is the jurisdiction with the lowest proportion of its identified schools receiving Title I funds. The 
apparent reason for this is that Kentucky bases its accountability system on a model of 
continuous improvement. ALL schools are expected to show progress relative to past 
performance and to established high standards. Since schools are evaluated based on their own 
record, relatively high-performing, low poverty (and thus non-Title I) schools in Kentucky are 
more likely to be picked up by the accountability system and designated as needing improvement 
than are such schools in systems emphasizing an absolute level of performance. 

Meanwhile, San Francisco and Chicago have moderate proportions of Title I schools 
identified by their local accountability systems. Here the explanation seems to lie more with the 
level of schooling than the identification criteria. In San Francisco, for example, of the six non- 
Title I schools that were identified for CSIP between 1993 and 1997, four were high schools. 
Proportionally, high schools in San Francisco as in other parts of the nation, are less likely to be 
receiving Title I funds than are elementary schools. In fact, only one of the 64 Title I schools in 
San Francisco in 1997-98 was a high school, while 54 were elementary schools. By contrast, of 
the elementary schools identified for CSIP during this period, 100% were Title I. 

Finding Two: There is considerable overlap between the schools identified for 
improvement under Title I and those identified in the jurisdictional systems. This overlap 
is not absolute, however, and varies considerably among the jurisdictions. 

Our next step in the analysis of the identification of schools was to focus in on the 
overlap among Title I schools identified by the jurisdictional systems and those identified for 
improvement in Title I. Our findings are displayed graphically in Figures 2a-2d and in tabular 
form in Table 2. Please note that there are no graph and no figures for San Francisco. At the 
time of this study, San Francisco was still in negotiation with the state about the criteria for 
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identifying program improvement schools in the district, so for 1997-98, there were no schools 
identified for program improvement*'. 

For the other four jurisdictions, it is possible to consider the degree of overlap among 
identified schools in two ways, both of which are indicated in Table 2. One way is to consider 
the overlap (that is, the intersection of schools identified by the jurisdictional and the Title I 
systems) as a proportion of jurisdiction-identified Title I schools. The question addressed by 
such an analysis is what proportion of Title I schools identified in the jurisdictional 
accountability system were also identified in the Title I system? Alternatively, one could 
consider the overlap as a proportion of program improvement schools (i.e., what proportion of 
schools identified for program improvement in Title I are also identified for sanctions in the 
jurisdictional system?). Taking New York City as an example, we see that in the first instance, 
74% of the Title I schools identified for SURR in New York City were also identified for 
program improvement in 1997. From this perspective, the overlap between the jurisdictional 
system and the Title I system is quite large. However, if we take program improvement schools 
as our point of reference, the overlap appears as a small sub-set; more specifically, only 12% of 
program improvement schools in New York City were also identified for SURR. 



" From statements made in March 1998 by Dr. Mary Welsh Byrd, Director of State and Federally Funded Projects 
of the San Francisco Unified School District. See footnote 17. 
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State and Title I Accountability: The Overlap 
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Figure 2b. 
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Local and Title I Accountability: The Overlap 
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Table 2. Identification of Schools by Jurisdictional and Title I Accountability Systems 
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In Chicago the situation is somewhat reversed, with only 68% of probation schools also 
on the state Title I improvement list (i.e., the State Warning List)*^ but 92% of Title I 
improvement schools on probation. In Maryland the situation is different still, with only 37% to 
40% of either group being identified by the other. Compare this limited degree of joint 
identification in Maryland with the total congruence of the two groups in Kentucky; i.e., in 
Kentucky 100% of program improvement schools are schools identified by the state system as in 
the improving, decline or crisis categories, while 100% of Title I schools identified as improving, 
decline, or crisis are also in program improvement’^. When only decline and crisis schools are 
considered (because of their similarity to probation or Reconstitution Eligible designations), they 
form a completely circumscribed subset of the larger group of Title I schools in need of 
improvement. Thus, 30% of schools in need of improvement were in decline or crisis while 
100% of decline or crisis Title I schools were in school improvement. 

How do we explain this significant variation among the jurisdictions? What does it mean 
when a Title I school is identified for improvement by one accountability system and not by the 
other? Understanding the reasons behind this variation among jurisdictions can provide insights 
into how the jurisdictions are approaching Title I accountability in the context of their own 
accountability structures. 

Finding Three; Where identification of schools by the jurisdictional and Title I 
systems is not congruent, the discrepancy stems primarily from differences in the criteria 
used for identification. These differences in criteria, in turn, seem to derive from several 
inter-related factors; differences in the purposes of the two systems, differences in who 
(i.e., which unit or level) identifies the schools, and general problems in implementation 
and administrative coordination. 

Our understanding of the reasons for the discrepancies in schools identified by the two 
types of systems is based on analyses of the specific policies in each jurisdiction and on 



As we will see, the district list for program improvement is another matter entirely (see Figure 3). 

The “improving category” is comparable to a “watch list” in many states. These are schools that failed to meet 
their improvement targets but did not decline in performance. By definition, Title I program improvement schools 
are all Title I schools falling in the improving, decline or crisis categories. 
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interviews with state and district personnel. (Summaries of the policies may be found in 
Appendix A; tables categorizing the criteria, support and consequences for Title I and 
jurisdictional accountability in each site are in Appendix B.) The most obvious and pervasive 
reason for the discrepancies in each case is that the two systems are using differing criteria. 

Chicago and San Francisco, for example, use completely different assessments in their 
local accountability systems than those used to determine adequate yearly progress for Title I in 
their states (ITBS vs. IGAP and CTBS vs. Stanford 9 respectively). Moreover, even where the 
same assessments are used, results may be defined and interpreted differently in the two systems. 
In New York, for example, the main criteria for SURR is that the schools are farthest from 
standard on the state assessments; the criteria for Title I improvement is that they are not making 
adequate progress on those same assessments. Finally, systems may differ in the additional 
criteria they include with the assessments. San Francisco, for example, bases its identification 
for CSIP on a 100 point index derived from 17 indicators, including student performance, need 
indicators, grade retentions, and staff stability. According to one local official, the system is 
neither standards- nor progress-based, both requirements for Title I accountability. While other 
district administrators would no doubt disagree with that characterization, there is no denying 
that the constellation of criteria used by San Francisco is unique to that jurisdiction and differs 
substantially from AYP in other parts of the state. 

While the details and specific stories vary by jurisdiction, the use of differing criteria can 
be traced to several underlying factors: differing purposes of accountability systems, differing 
levels of governance responsible for identification, and implementation issues. 

Differing purposes/tareets of accountability 



The most obvious example of differing purposes for Title I and jurisdictional 
accountability is that of New York, where Title I SURR schools (those farthest from state 
standards) are largely a subset of the larger group of schools not making adequate yearly 
progress. This pattern does not derive from a decision to relate the systems in that way, 
however, as is the case in Kentucky. In Kentucky, Title I schools in decline or crisis (the 
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designations which bring to bear the Distinguished Educators and the threat of more serious 
sanctions) are the subset of the total group of Title I schools in need of improvement, but they 
are so as part of a unified, coherent, and multi-level system. In New York, there is no clear 
relationship between SURR and Title I accountability; it is simply that the state has chosen to 
target the schools in most severe straights for state intervention. Meanwhile, Title I 
accountability remains a separate system, targeting schools on a broader basis. 

Differing levels of governance 



The most common contributing factor to the differing criteria among systems is the level 
of governance at which identification is defined and made - that is, the state vs. the local district. 
Again, Kentucky avoids this problem - this time by essentially cutting the local units out of the 
decision-making process, as all identification is made by the state, based on state tests and state- 
defined criteria. 

By contrast, local districts in New York, Maryland, and California play a considerable 
role in identifying which Title I schools are in need of improvement. This role is consistent with 
federal guidelines for Title I: while states define adequate yearly progress, LEAs identify schools 
in need of improvement. Moreover, the state may allow LEAs to add local assessments or 
develop their own assessment systems to be incorporated into the definition of AYP and 
designation of schools needing improvement. 

In these three states, the district role in identifying Title I schools in need of improvement 
leads to some interesting patterns. Consider Maryland (Figure 2b) as a more pronounced case, 
where the combination of differing levels of analysis and differing criteria at the local level 
results in a relatively small portion of schools being concurrently identified by both 
accountability programs. In the first place, the Maryland SEA only recommends that districts 
identify for improvement those Title I schools with a negative change score on the state- 
determined School Performance Index (SPI). In practice, while LEAs place strong emphasis on 
MSPAP and the SPI, they also consider other factors such as mobility or relative position among 
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the county’s schools. This means that the criteria for Title I identification will differ from county 
to county. No such variation occurs in the state accountability system. 

Secondly, all counties select at least some Title I schools for improvement, generally the 
schools most in need relative to others in the district. In contrast. Reconstitution Eligible 
schools are concentrated almost exclusively in Baltimore City (79 of 90) and to a lesser extent in 
Prince George’s County (9 of 90). This is because the Reconstitution Eligible designation is 
reserved for those schools that are at the lowest levels of performance (and declining) relative to 
all schools in the state. Since most such schools are in Baltimore City and PG County, nearly all 
(88 out of 90) RE schools are located in these two districts. This helps to explain the graph in 
Figure 2b, as Title I improvement schools in other counties fall outside the RE circle on the 
graph. In addition, Baltimore City has chosen to target only a subset of its RE Title I schools for 
Title I program improvement. This explains the large area of the Reconstitution Eligible circle in 
Figure 2b that lies within Title I but outside program improvement. The end result is that only 
37% of school improvement schools are also Reconstitution Eligible and only 40% of 
Reconstitution Eligible Title I schools are designated as improvement schools. 

Maryland is a case where the locally determined Title I accountability system is only 
somewhat congruent with the statewide system. San Francisco and Chicago represent a different 
type of local-state disjuncture. In the case of these two cities, the jurisdiction- wide 
accountability system is local, based on local needs, goals, and context. The criteria for these 
systems grew out of those local contexts and exists independently of Title I accountability. In 
San Francisco, the system was created to respond to a court-ordered desegregation consent 
decree. It pre-dates the 1994 reauthorization of Title I, and district administrators have been 
negotiating with the state to accept the local system for Title I requirements. Similarly, in 
Chicago, assessments used for probation and reconstitution determination differ from the state 
tests used for AYP. While the overlap of schools identified locally for probation and schools 
identified by the state Title I office is considerable, it is not absolute. Moreover, because 
Chicago has a well-developed local improvement program for probation schools, they have 
chosen to define higher-performing “borderline” schools for improvement under Title I. Defined 
as a sort of “division of labor” between the local accountability system and Title I accountability. 
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the accommodation results in the targeting of a much larger number of schools for assistance 
efforts. It also results in the set of “program improvement” schools being completely outside the 
set of probation schools (see Figure 3). 




Implementation issues 

A final explanation of differing criteria and results rests in two sets of implementation 
issues. One issue is that of timing. On the one hand, Title I accountability is meant to be phased 
in as other provisions are phased in. For this reason, the definition of AYP in most jurisdictions 
has been in flux over the past few years'"*, and sanctions for persistent failure to improve have yet 
to be determined (see below). On the other hand, some state and local systems pre-date Title I 
accountability - that is, they have an institutional history that may not be entirely compatible 
with the specific provisions of Title I. 

Added to the timing issue is a second set of implementation problems which center on the 
continuation of fragmented, stovepipe Title I administration in most jurisdictions. While there 



‘“'in fact, the Title I guidance suggests that states use the transitional period to experiment with different definitions 
of AYP. 
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has clearly been progress in these five jurisdictions in making the systems more coherent, there 
exists, as indicated in Section I, a continuing division of responsibility and knowledge between 
Title I and the regular program. 

Tn either case, the result is that schools mav be subject to differing test s and differing sets 
of criteria for Title T and for their local/state accou n tability systems. To the extent that the 
legislation and guidance specify a separate system a n d criteria for Title I, the federal govemmsnt 
appears to be contributing to such dua l systems. 

Sanctions and Assistance for Low-Performing Schools 

Findings 4 and 5 focus on the consistency and cooperation in sanctions and support 
available to schools identified in the jurisdictional and Title I programs. 

Finding Four: Lack of specified consequences for Title I schools that fail to improve 
seems to contribute to a relatively low saliency for Title I accountability in these 
jurisdictions. 

Although Section 1116 of ESEA requires districts and states to take corrective action in 
program improvement schools that fail to make sufficient progress after three years, significant 
consequences are not expected to take effect until after the next authorization of ESEA, or at 
least not before 2000. As a result, many jurisdictions are still in the process of defining what 
those consequences and corrective steps will be. 

When we asked respondents in these five sites about consequences, none were mentioned 
apart from the requirement that schools identified as in need of improvement develop a school 
improvement plan. Either the respondent did not know that consequences were required or they 
did not mention them because at the time they were not yet defined and in place. In either case, 
we had no indication that the potential consequences called for in the legislation had any 
salience, either for the administrators or for the schools. 
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By contrast, the jurisdiction-wide systems of accountability in these five sites had very 
clearly delineated consequences. Those consequences varied for jurisdiction to jurisdiction, as 
indicated in section 1 and the tables in Appendix A. But in all cases, the potential for 
reconstitution, closure, or takeover loomed as a threat over low-performing schools; what’s 
more, the specific nature of the threat in that jurisdiction was well known to the individuals with 
whom we spoke. We concluded that the consequences outlined in the jurisdictional systems 
carried with them a saliency that was lacking in the more diffuse (and perceived low stakes) 
Title I systems. 



This difference underscored our impression of separate systems in all but Kentucky (and, 
at the state-level only, Illinois). It is unclear, how or when the Title I consequences will be 
defined. With the exception of Kentucky and Illinois, there is no automatic movement of Title I 
schools into the higher sanctions of the jurisdictional systems. 

Finding five: Support provided to low-performing schools (identified by either 
Title I or jurisdictional accountability systems) varies substantially from jurisdiction to 
jurisdiction and often from school to school. 

Our data on specific support to Title I schools are limited, but overall demonstrate the 
following patterns: 

1 Much of the support in each jurisdiction i s focused on helping schools develop and 
implement an improvement plan that is consisten t with relevant requirements and responsive to 
particular school needs . 

The most promising example of the planning process and assistance seemed to be m 
Kentucky, where consolidated plans were instituted at all levels in the 1997-98 school year. 
Schools identify needs based on assessment results, analyze causes of problems, and specify 
actions to be taken. Title I, along with other resources, is allocated strategically to address the 
needs of the school and district, not to establish a separate program. In addition, a major focus of 
the Distinguished Educators working with schools in decline or crisis has been to assist them m 
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using data to assess needs and in developing plans that will align curriculum and instruction with 
state standards. 

Chicago and San Francisco also provide schools with assistance in the planning process. 
In Chicago, this assistance comes in large part from the probation manager assigned to each 
school on probation. An important part of the probation manager’s role is to help develop the 
corrective action plan, followed by monthly monitoring of the plan’s implementation. 

Similarly, all CSIP or reconstituted schools in San Francisco have an assigned liaison 
from the central office, whose job is to serve as an information conduit and support provider. In 
the case of the five reconstituted elementary and middle schools, this liaison is the principal of a 
successful reconstituted middle school. This individual, who works with the schools both 
individually and collectively through regular meetings of the principals, identified four central 
foci in his work in 1997-98. One of these was helping with the school improvement plans and 
another was budgeting. In both areas. Title I was an important point of discussion and 
assistance. One indication of the importance placed on Title I in this group came in just the 
second meeting of these principals, held before the start of the 1997-98 school year. At this 
meeting, the district Title I coordinator presented information on Title I resources, regulations, 
and options, offering her assistance to the identified schools. She reported to us that this was the 
first time she had been invited to attend such a meeting of school principals in her several 
decades as Title I coordinator. 

While the planning process is also emphasized in Maryland and New York policies, no 
specific or consistent forms of assistance in planning were mentioned by respondents from these 
jurisdictions'^. 

It should also be noted that anecdotal information from each of the jurisdictions, as well 
as our case studies of reconstituted schools, suggests that the effectiveness of the assistance 
provided for the planning process varies considerably from school to school, depending upon the 
capacity of the specific assistance provider and the relationship established with the school 

See note 3 for changes in Maryland’s assistance and improvement planning policy. 
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leadership. A more thorough investigation of that variation, however, was beyond the scope of 
this study'®. 

2. A variety of other forms of professional development and capacity building ar_e 
available to schools identified through the accountability systems. The primary source o f the 
support, however, is the jurisdiction, with Title I assistance appearing somewhat peripheral and 
sporadic. 

Again, Kentucky and Chicago have the most extensive support - in Kentucky through the 
Distinguished Educators (DEs) and in Chicago through the external partners (EPs). In both 
cases, the primary support is that provided by the jurisdiction- wide system; it is also site-based 
and extensive. The DEs in KY work with schools over a two-year period. The external partners 
have annual contracts, but schools are encouraged to stay with their external partners for multiple 
years. In Kentucky, the DEs have received common training and take a fairly common approach 
to schools, particularly regarding standards and assessments. Chicago’s external partners, by 
contrast, represent a variety of approaches and services from restructuring into small schools to 
curriculum development to coaching with individual teachers on pedagogical strategies and basic 
skills instruction. In neither case was there much specific mention of Title I in the context of 
these assistance efforts, though in Kentucky ESEA regional assistance centers are notified when 
Title I schools enter decline or crisis. Meanwhile, CPS officials specifically downplayed the 
usefulness of assistance from the regional comprehensive assistance center or school support 
teams, primarily because of their limited experience with Chicago’s urban context. 

This is not to say that Chicago does not have specific assistance for schools in need of 
improvement. Part of the rationale for the decision to designate “borderline” schools as Title I 
“program improvement” schools while leaving probation schools to local efforts was to make 
more rational use of assistance available through Title I resources. Thus, CPS has a separate 
strategy of program improvement for non-probation, low-performing Title I schools that includes 



Several studies of such assistance are cunently underway. One is an OERI field initiated study of School 
Improvement Plans in schools on probation! or a similar designation) in three jurisdictions. That study is housed at 
the University of Maryland and is under the leadership of Rich Mintrop and James Cibulka. 
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administrator training and degree programs as well as professional development activities for 
teachers. 

Two of the most intensive examples of assistance to identified schools occurred in NYC. 
One of these was the assistance offered PS 198 by the leadership of Community District 2, and 
the other was the capacity building efforts of the Chancellor’s District, composed entirely of 
SURR schools. These two cases share many characteristics. In both, assistance was multi- 
faceted and multi-level, involving coordinated and persistent efforts from multiple providers for 
both administration and teachers. Moreover, both are examples of the district leadership 
assuming responsibility for the success of identified schools. Both placed primary emphasis on 
developing the capacity of school leaders and teachers to engage in high quality instruction. 
Both worked closely with teacher union leaders, and in both cases, all involved schools were 
high poverty. Title I schoolwide project schools. Yet these two examples also demonstrate an 
important difference. In District 2, the strategy used for PS 198 was simply an extension and 
intensification of the existing approach to school improvement employed throughout the district. 
By contrast, the Chancellor’s District represents a change in governance, initiated precisely 
because the existing strategies of the relevant community school districts were judged 
insufficient to ensure success of their SURR schools (see profiles below). 



O 

ERIC 



40 



52 



Profile: PS 198 and Community School District 2 

PS 198, in Manhattan’s Upper East Side and Community School District 2, was 
placed on the SURR list in 1996 when only 26% of its third-graders met the 
state’s minimum level in reading achievement. The identification brought 
immediate response from the district leadership (“Not in my district!”) and 
mobilization of district resources. The approach to assisting the school was 
consistent with that used throughout the district: teacher learning is the key to 
student learning. Emphasis was therefore on professional development. Two 
highly successful “distinguished teachers” were recruited from elsewhere in the 
district and assigned to work full-time with four newer teachers in the school, 
rotating among them in eight-week intervals, modeling, coaching, and co- 
teaching all day, every day. Other teachers in the school engaged in extensive 
professional development opportunities two or three times a week, and the 
principal met regularly with other principals to discuss instructional 
improvement strategies. In addition, the deputy superintendent visited the school 
monthly, observing classrooms and monitoring the progress of individual 
teachers and students. The effectiveness of these visits rested on the relationship 
established between the deputy and school personnel over a number of years. 
Title I resources were used to provide additional teacher support in the regular 
classrooms, and students had opportunities for extended day and summer 
learning. The central focus of all these efforts was on literacy. Those efforts 
paid off: in 1996-97, 74% of PS198 third-graders met the state’s standard, and 
the school has subsequently been removed from the SURR list. 



BEST COPY AVAILABLE 



Profile: The Chancellor’s District for instruction 

Like District 2, the Chancellor’s District (District 85) in New York City, has 
focused its strategy on building the instructional capacity of principals and 
teachers. The twelve schools in the district, all on the SURR list, are among the 
lowest performing elementary and middle schools in the city. In 1996, 
Chancellor Rudy Crew removed authority for these schools from their respective 
districts, placing it instead in the hands of a specially created governing unit. 
Pulling together a bunch of “failing schools” into one district did not 
immediately engender confidence, however. Rather, that became the job of 
Supervising Superintendent Barbara Byrd-Bennett, who stressed to the schools, 
“We’re in this together. . .1 believe in you!” Byrd-Bennett’s cheerleading was 
backed up by concrete assistance that allowed the school leaders to believe in the 
superintendent’s words and in her faith in them. The assistance included 
everything from immediate building repairs to smaller classes to extended days; 
from developing core leadership teams in each school to monthly principal 
institutes focused on instruction to school-based instructional coaching; from 
promoting an absolute focus on literacy to systematic disaggregated data analysis 
to assistance in recruiting teachers. The intensity of support paid off in District 
85 as it had in District 2. The twelve schools of the Chancellor’s District posted 
the largest one-year gains on the city’s reading and mathematics assessments in 
1998 of all the NYC districts, with the reading gains being twice those of any 
community school district. 
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Section III. Discussion and Recommendations 



How should we interpret these findings on the congruence of Title I accountability 
measures and those of states and local districts? 

We would stress again the caveats we expressed earlier: in this study we are looking at 
only a few jurisdictions and those with established systems. We are not sure what our findings 
mean for states that had not already established or were moving toward similar accountability 
systems at the time ESEA went into effect. Nonetheless, with respect to these five jurisdictions, 
several conclusions are worth noting. 

l^^ll of the jurisdictions were attempting to comply with the requirements of Title I 
accountability and were doing so primarily by trying to fold Title I accountability into 
jurisdiction-wide systems in some wav . 

This was most easily and completely accomplished in Kentucky, which had an 
established, multi-level system of school accountability based on progress toward state standards 
and assessments. Since there already were standards in place and the state used a model of 
continuous improvement, Kentucky’s system was completely compatible with Title I legislation 
and guidance. Moreover, no district variation exists, as each LEA uses exactly the same system 
as the state. 

Other states, Maryland, New York, Illinois — and now California, used the same 
assessments as the basis for both Title I and the statewide accountability system. However, 
where districts in those state incorporated local assessments or other measures into Title I 
identification criteria, the result may be the targeting of different schools for Title I and statewide 
accountability 

With regard to the two district-based accountability systems in this study (San Francisco 
and Chicago) we also see an attempt to fold in Title I, in this case, by negotiating with the state 
to use the local accountability criteria as the definition of AYP for their schools. Chicago has 
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experienced some success in this effort resulting in a division of labor: the Chicago Public 
Schools Office of Accountability provides assistance and monitoring for probation schools 
(many of which are on the state Title I list) while the local Title I administrator provides 
assistance for borderline and ^Temediation” schools, as part of the locally defined Program 
Improvement” program. San Francisco was having more difficulty in their negotiations with the 
state, but this appeared to be moving toward resolution at the time of data collection . 

2. Despite attempts to make the systems coherent. Title I schools often appe ar to be 
siihject to dual systems of accountability. Thus far in the implementation of ESEA, the existence 
of these dual systems does not seem to be a major problem, but this may be due in part to the fact 
that consequences for title I have not yet really kicked in. Title I accountability and corrective 
action, therefore, do not have high saliency in these sites and thus do not divide school or 
administrators’ attention. It is conceivable that if the dual systems continue, and if corrective 
actions (such as alternative governance and reconstitution) are applied to Title I schools based 
upon current definitions of AYP, schools could face similar high stakes consequences coming 
from differing sources using differing criteria. We could find no clear advantages to such a 
situation and can anticipate substantial disadvantages in the long run. Overall, this pattern of 
dual accountability seems to run counter to the general intent of ESEA to foster coherence in 
educational programming for all students and schools. Moreover, to the extent that the 
legislation calls for a specific system of accountability for Title I schools, one could argue that it 
contributes to the creation of such dual systems. 



^ The problem of low performance, accountability, and assistance for schools is 
particularly acute in urban areas. Consider the following findings: in New York City in 1998, 
552 Title I schools were identified for program improvement and 96% of state SURR schools 
were also in New York City. In Maryland, 88% of Reconstitution Eligible schools are in 
Baltimore City, making up half of the total number of Baltimore schools. In Chicago, one-fifth 
of the schools had fewer than 15% reading at grade level on a test of basic skills in 1996, and in 



” No resolution emerged, however, and as of March 1999 no plan for identifying program improvement schools in 
San Francisco had been approved by the state. Nonetheless in summer 1998 the Title I office designated fourteen 
schools for program improvement for the upcoming school year and a preliminary list for 1999-2000 shows 32 
candidates for program improvement assistance. 
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San Francisco, ten of its 114 elementary, middle and high schools have been reconstituted in the 
past four years, while almost a quarter have been identified for CSIP. 

Recognizing this high need, urban districts have been developing their own 
accountability systems - or in the case of Baltimore City having accountability thrust upon them. 
Any Title I system instituted, therefore, must recognize the special problem of accountability and 
performance in the urban areas. 

Recommendations 

Neither our intent nor our charge in this study was to develop a comprehensive set of 
recommendations for improving accountability provisions in Title I. Nonetheless, two central 
recommendations from this work are almost unavoidable; they concern the need to guard against 
the establishment of dual (jurisdictional and Title I) systems of accountability and the need to 
greatly improve the assistance provided to schools in need of improvement. We have also 
included a third set of suggestions for strengthening the implementation of several elements 
already in the legislation or guidance but which are too rarely implemented by the states. 

1) Fostering coherence 

Based on our findings and analysis for this study, our central recommendation concerns 
the need to foster greater coherence in school accountability requirements. 

ftn r^thoiizin^SEA, Congress should remove or revise those 

Iprocesses or provisions that foster the development of a dual system of 
fcchool accountability for Title I schools. « 
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More specifically, we suggest the following: 



• To receive Title I funds, states must have in place (by a given time) a system for 
holding schools and districts accountable for all children making progress toward 
meeting challenging state standards. This system must apply to all schools, including 
those covered bvThe act. Once that system is in place, no separate performance- 
based accountability system should be applied to Title I schools on the basis of their 
Title T status. 

• In the interim (while statewide systems are being developed), a temporary 
accountability system for districts and schools receiving Title I funds should be 
employed, similar to the current provisions for accountability under ESEA. 

• If districts develop their own performance-based accountability systems, these should 
replace or be incorporated into the state systems so that schools are not subject to 
multiple accountability criteria and processes stemming from implementation of Title 
I provisions. 

2) Building Capacity 



Based both on this investigation and on the parent study, we believe that limited 
instructional capacity is a critical factor in persistent low performance and we see a need to 
vastly improve assistance provided Title I schools in need of improvement. 



;|The reauthorized ESEA should place greater emphasis on ensuring that 
Ischools in need of improvement receive appropriate, consistent, and 
lintensive assistance from districts and states. 



More specifically, we suggest the following: 



• Changes to technical assistance provisions of ESEA should be based on a 
comprehensive review of the technical assistance and professional development 
currently provided thorough Title I and ESEA (a task clearly beyond the scope of this 
study). 

• Assistance under ESEA should be consistent with and integrated into other local and 
state-based assistance efforts, rather than under separate authority and administration. 
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• Districts should be held accountable for ensuring the progress of low-performing 
schools in their jurisdiction. We recommend that LEAs be required to submit a plan 
for assisting Title I schools identified in need of improvement along with the 
improvement plan submitted by each school. The LEA plan should include an 
examination of district policies that contribute to school failure and a strategy for 
addressing them. 

• Greater resources should be targeted toward ensuring the improvement of teaching 
and teacher quality in low-performing Title I schools — including allocating more 
resources for professional development of teachers, administrators, and 
paraprofessionals and for the recruitment of more qualified staff into these schools. 
Emphasis must be placed on instructional improvement consistent with state 
standards in these assistance efforts. 

3) Strengthening criteria and consequences 



pjJased^ this and the parent study, we suggest that several aspects of 
Title I accountability guidance regarding criteria and consequences be 
istrengthened, either by incorporating them directly into law or by 
■working closely with states to foster their implementation. 



More specifically, we recommend that the accountability systems developed by the 
states, in accordance with Title I provisions: 



• Incorporate multiple criteria for decisions about schools and districts. While current 
law requires multiple measures in final state assessment systems, in practice many 
jurisdictions rely on a single test for identifying schools in need of improvement. 

This pattern weakens the legitimacy of the system and raises issues of consequential 
validity when used for high stakes decisions such as probation and reconstitution. 

• Disaggregate results by relevant sub-groups or performance levels in the school and 
require improvement for sub-categories of students as well as for the school as a 
whole. One disturbing pattern noted by a number of observers and participants in 
these systems is a tendency for some schools to concentrate improvement efforts on a 
certain segment of the student population, such as those on the cusp of a proficiency 
category or those most easily reached. By doing so, school personnel may increase 
the relevant measure of aggregate school performance (proportion of students 
scoring at or above a certain cutoff in some jurisdictions, the mean level of 
achievement in others) and thus be removed from threat of sanctions. Such practice, 
however, does not help to ensure progress of all students in the school toward the 
state and local standards. Requiring improvement among relevant sub-groups of 
students would help to ensure equity in accountability provisions. 
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Finally, we support the incorporation of corrective actions into the school 
accountability systems. We have seen in these jurisdictions how sanctions seem to 
increase the saliency of an accountability system and to focus attention on 
improvement targets. However, we would also urge that requirements for corrective 
actions allow for some degree of flexibility in their design and use, based on local 
evaluation of the causes of poor performance in a given school and determination of 
the most effective means of addressing those causes. We also suggest, as a way of 
increasing district accountability, that districts provide a clear rationale for how the 
given action responds to the needs of a particular school and its students. 
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California 

Adequate Yearly Progress and State Accountability (AYP) 

The definition of adequate yearly progress has been in flux for the past few years. 
1996-97 was the first year the state imposed requirements on AYP. The state required all 
schools to have 40% of its students at grade level to satisfy adequate yearly progress. However, 
the schools were not advised of the criteria until after the school year was out. To meet the 
state’s demand the districts had to reconstruct scores from the assessments they did perform to 
evaluate the grade level status of their students. Due to the late information and lack of 
appropriate assessments, several districts did not report any information on the adequate yearly 
progress of their schools. 

In April 1998 the state established a new set of accountability requirements. The state 
indicated that the Stanford 9 assessment must be a component in the evaluation of grade level. 
With this exam the 50*'’ percentile will be the cut off for proficient. However, schools are 
allowed to consider alternative assessments for students that score below the 50*'’ percentile. 
Districts could also establish their own measures as long as they were approved by the state. 

Additionally, the state decided to include an improvement component to the 
accountability system. They established 90% meeting or exceeding grade level on assessments 
based on standards as the target to be met in eight years. Schools are required to meet the 
average yearly improvement necessary to reach the goal in eight years. A member of the state 
department of compensatory education indicated that the interim state assessments (Stanford 9) 
need to be changed. This year the Stanford 9 was used with the 50*'’ percentile mark as grade 
level. Schools were still offered the option to use other assessments to evaluate the grade level 
of students that fell below the 50*'’ percentile. The state published guidelines for the use of 
alternative measures. Next year the state will likely use the augmented SAT/9 with some 
alignment to state content standards. She also remarked, “It is likely that our state board of 
education will assign NAEP proficiency levels to scores on the augmented SAT/9.” 

/ 



Support for Program Improvement Schools 

Schools can apply for funding or receive technical assistance from a variety of sources. 
First, schools can apply for Title 1 set aside funds for school improvement. To be eligible, the 
school must submit a plan for improvement. Schools can receive assistance in professional 
development from the regional network, a statewide system of school support. Also, schools can 
receive assistance from the federally supported Comprehensive Assistance Center. 
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Illinois 

Adequate Yearly Progress and State Accountability 

Illinois instituted the Academic Early Warning List as a system of school improvement 
for all schools in the state. This list identifies schools in two categories: “Does Not Meet”, also 
referred to as Level 1, and “Declining.” If a school has more that 50% of all Illinois Goal 
Assessment Program (IGAP) student scores in the school not at standard for two years 
consecutively, the school is identified in the “Does Not Meet” category. If a school increases the 
number of below standard students by 20% or more three years following the benchmark year, 
they are classified as “Declining.”(Note: schools can be identified as declining even though 
more than 50% of the school’s students meet standard.) 

Once identified as a Level 1 “Does Not Meet” school the state requires the school to meet 
or exceed standards, or continue to make adequate yearly progress until the goal is met. Schools 
that meet the standard are removed from the warning list. Otherwise, the school will remain on 
the list as long as the school makes adequate yearly progress. If a school fails to meet the 
standard and does not make adequate yearly progress after two years on the warning list, the 
school is placed on the academic watch list. 

Schools classified as “Declining” can be placed on the academic watch list if the school 
has declined such that more than 50% of its students do not meet standard for two consecutive 
years. Declining schools lose this designation when the percent of IGAP scores at or above 
standard equals the average percent of scores at standard for the previous 4 years. 

Adequate yearly progress is defined in terms of progress toward a goal of 50% IGAP 
scores at standard over five years. Therefore, adequate yearly progress is satisfied if a school 
increases the percent of IGAP scores meeting standard by one fifth of the difference between the 
benchmark and 50%. Despite this seemingly strict formula, a representative in the state board of 
education indicated that in practice meeting adequate yearly progress is less strict. She explained 
that the state performs a rather subjective evaluation of progress. Schools generally just need to 
report their progress and what the school has done to raise scores. 

Probation/Reconstitution policy in the Chicago Public Schools (CPS) also relies heavily 
on test scores. However, Chicago also examines attendance rates and drop out rates in high 
schools. Identification of failing schools is made at the district level. Schools are first identified 
for remediation based on IGAP scores. The second level of identification is probation. The 
basic criterion for probation is performance on the Iowa Test of Basic Skill (ITBS) and Tests of 
Academic Proficiency (TAP). If less than 15% of a school’s students achieve national norms, 
the school is subject to probation. The district also considers attendance rates, dropout rates, and 
IGAP scores for borderline schools. Finally, schools that the district determines have not 
progressed, or declined during probation are identified for reconstitution. 

State and CPS criteria for entering the accountability process appear to be aligned in two 
respects. First, IGAP scores are used to identify schools for academic warning and remediation. 
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Second, CPS recognizes removal from the state academic watch list as one criterion for removal 
from the probation list. 



Support for Program Improvement Schools 

The state requires all schools on the academic early warning list to develop a School 
Improvement Plan (SIP) that must be submitted to the local school board or local school council 
(LSC) and the state superintendent. To assist schools in this process the state board of education 
provides a staff member to work with the school. In Chicago the probation member serves as the 
contact with the state instead of an Illinois state board of education (ISBE) staff member. 

Schools also qualify for targeted school improvement funds from the state. Finally, every two 
years a panel of outside educators, parents, and community members will evaluate the schools’ 
services, the impact of these services on student learning, student work, and achievement data. 
This process is known as a Quality Assurance External Review. 

Title I schools that fall on the Academic Warning list receive all of these services as 
program improvement schools. There was very little discussion of school support teams (SSTs). 
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Kentucky 

Adequate Yearly Progress and State Accountability 

Title I accountability is based on the statewide accountability system. All Title I schools 
identified as improving, improving category 2, decline, and crisis are identified as improvement 
schools. Until this year only crisis schools had staff placed on probation. 

The schools are rated by the accountability index. The index score of 100 implies that 
100% of students are proficient, there was no retention, no drop-outs, and 100% of students 
successfully transitioned to adult life. Clearly, 100 is the optimal goal. In reality, schools are 
indexed at the beginning of a biennium and this index is identified as the baseline. Schools then 
identify an improvement goal. Schools that meet or exceed the improvement goal are identified 
as successful or reward schools. Schools that do not meet the goal but improve are identified as 
improving or improving category 2. Schools that decline are identified in decline or crisis. Prior 
to the most recent legislation, schools that fell more than five points on the index were identified 
in crisis. 



Assistance to Improvement Schools 

All schools in decline, in crisis and improving category 2 receive the assistance of state 
Distinguished Educators (DEs). DEs provide assistance in developing and implementing the 
SIP. Also, these schools qualify for Commonwealth School Improvement Funds. Title I schools 
receive all of these services but also regional school support teams are notified of their status. 
These SSTs assist in developing the SIP and improvements. However, their involvement is less 
intensive that DEs. Other than the SST services. Title I schools receive the same assistance as 
non-Title I schools. 
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Appendix A: Accountability Policies— Maryland 



Maryland 

Adequate Yearly Progress and State Accountability 

The state relies on the School Performance Index (SPI) to track the success of schools. 
Elementary Schools are at standard when on average 70% of the school’s students score 
satisfactory on the content exams and attendance is at 94%. The standards for a middle school 
are 94% attendance, 70% satisfactory on the Maryland School Performance and Assessment 
Program (MSPAP), and 95%, 85%, and 90% on reading, math, and writing respectively on the 
Maryland Functional Test (MFT). The standards for a high school are 94% attendance, 97% 
retention, and 95%, 80%, 90%, 85%, 90% scoring satisfactory on the MFT reading, math 
writing, citizenship and all 1 grade exams respectively. It should be noted that attendance, 
retention, and average outcomes on exams are weighted equally in the SPI. 

The state suggests that all districts identify schools at standard and schools showing 
substantial or sustained progress toward the standards as meeting adequate yearly progress. The 
state also recommends that all schools with a negative change in the index should be identified 
for school improvement. However, Local education agencies (LEAs) identify schools in their 
districts. Discussions with some LEAs revealed that not all schools with a negative change index 
are identified. Also, some schools that show a positive change index may be identified if the 
school is particularly needy. In identification LEAs appear to place a strong emphasis on the 
MSPAP and SPI. However, they may choose other variables such as mobility, relative position 
among the county’s schools, or severity of decline. The limited sample suggests that a uniform 
standard across the state does not exist. 

Baltimore City schools focus on the School Performance Index as the basis for 
identifying schools that fail to meet adequate yearly progress. The director of compensatory and 
funded programs for the Baltimore City Public Schools stated that “ All schools with a negative 
SPI are identified as potential needs improvement sites.” Furthermore, he indicated that only 
schools with a negative SPI are considered potential “Title I schools in need of improvement.” 
However, schools with a negative SPI may not be identified if indicators from other achievement 
data or special circumstances suggest that the school should not be identified. 

Prince George’s County’s process of identifying schools for Title I school improvement 
makes direct reference to the state’s reconstitution policy. The primary criteria determining 
adequate yearly progress are the MSPAP and the Comprehensive Test of Basic Skills. All 
Reconstitution Eligible schools are identified for Title I school improvement. Schools with a 
negative SPI are identified for improvement if they are identified as Reconstitution Eligible, 2L‘ 
Century, Alert, or Title I schools not meeting adequate yearly progress.. 

Identification as a Reconstitution Eligible school is made at the state. Schools that may 
be identified as Reconstitution Eligible are below standard and declining even after developing 
and implementing a SIP. Currently, only 90 schools are identified as Reconstitution Eligible 
schools. 
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Support for Identified Schools 

Apparently, districts in the sample discussed above help schools develop and implement 
SIPs. St. Mary’s county also provides resource people to help in the schools. Counties such as 
Anne Arundel have identified a county alert list on which all the improvement schools fall. All 
schools on this alert list receive assistance. There was almost no talk of school support teams. 
The former director of compensatory programs explained that the state is developing a new 
school support team comprised primarily of retired teachers and administrators. However, this 
SST is not yet established and is only intended to work with schoolwide programs whether they 
are improvement schools or not. 
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Appendix A: Accountability Policies — New York 



New York 

Adequate Yearly Progress and State Accountability 

In 1996 the state targeted 2001 as the year all schools should reach the regent’s standard. 
This standard is 90% of students in a school reaching the state benchmark. Given this state 
target, adequate yearly progress is defined as progress to reduce the gap between the 1996 mark 
and 2001. Schools with base line in 1996 and 1995 are expected to reduce the gap by 20% each 
year beginning in the 1996-97 school year. Schools with a base line fi"om 1997 must reduce the 
gap by 25% each year. All schools that achieve the 90% standard are expected to maintain the 
standard to satisfy adequate yearly progress. A publication from the Division of Assessment and 
Accountability states that “ schools which fail to meet their adequate progress target or fall 
below 90% threshold are to be identified in need of improvement.” 

The Schools Under Registration Review (SURR) process also targets 90% of students 
achieving the state standards. However, the state also places a minimum requirement on drop 
out rates, and considers the whole school environment. First, the state expects that the drop out 
rate be no higher than 5%. Second, the state identifies schools with a poor learning environment. 
Three elements contribute to the state’s definition of a poor learning environment. These 
elements include 1) poor performance on state standardized tests, 2) parent complaints to the 
state, 3) conditions such as, high violence, excessive absenteeism and suspension, or high 
percentage of uncertified teachers. Schools identified for SURR are those furthest fi"om the 
minimum standards or schools identified as a poor learning environment. 

SURR and Title I accountability processes incorporate the same state assessments. 
However, New York City (NYC) uses local assessments for Title I identification. Since SURR 
schools are identified by the state, local assessments are not incorporated. 



Support for Program Improvement Schools 

Title I improvement schools receive assistance from the community school district and 
NYC Title I school support teams (SSTs). However, SSTs focus their attention on schoolwide 
programs. Schoolwide programs identified for improvement receive a very high priority for 
intensive assistance. This assistance comes in the form of: 

1. Schoolwide planning 

2. Developing assessments aligned with state standards 

3. Capacity building 

4. Intensive on site assistance 
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Appendix B: Comparison Tables for Accountability PoIicies—IlHnois 



Illinois 

STATE IDENTIFICATION 





Academic 
Warning List 


Declining 

Schools 


Academic 
Watch List 


Criteria 


Less than 50% of 
students in the school 
meet state standards. 


The percent of 
students meeting 
standard declined by 
20 or more percent 
over three years. 


Failure to meet 
adequate yearly 
progress for two 
consecutive years 


Consequences 


Required to develop a 
plan for improvement. 
Schools are expected 
to make adequate 
yearly progress or 
meet the 50% level. 


Required to develop a 
plan for improvement. 
School is expected to 
raise the percent of 
students meeting 
standard to a level 
equal to the average 
level for the previous . 
4 years. 




Support 


1 . State board 
provides a staff 
member to assist 
in developing 
improvement 
strategies. 

2. Participation in a 
Quality Assurance 
External Review 


1 . State board 
provides a staff 
member to assist 
in developing 
improvement 
strategies. 

2. Participation in a 
Quality Assurance 
External Review 
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Appendix B: Comparison Tables for Accountability Policies— Illinois 



Illinois cont. 

CHICAGO IDENTIFICATION 





Remediation 


Probation 


Reconstitution 


Criteria 


Low test scores on 
IGAP 


Less than 15% 
meeting national 
norms on ITBS and 
TAP, and failure to 
improve under 
remediation 


Failure to improve 
after two years on 
probation 


Consequences 


Must develop a plan 
for school 
improvement 


Must develop a plan 
for school 
improvement 


1 . School is re- 
staffed. 

2, Must develop a 
plan for school 
improvement 


Support 


Assistance developing 
the improvement plan 


Provided a probation 
manager and external 
partner to assist in 
developing and 
implementing the 
improvement plan 


Provided a probation 
manager and external 
partner to assist in 
developing and 
implement the 
improvement plan 
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Appendix B: Comparison Tables for Accountability Policies—Kentucky 



Kentucky 





Improving 


Improving 
Category 2 


Decline 


Crisis* 


Criteria 


Improvement was 
less than the goal 
for the present 
cycle. 


The school’s 
improvement in the 
present cycle was 
less than the goal 
AND the school 
was identified 
improving, or 
decline in the 
previous cycle. 


The school’s index 
fell below the 
benchmark for the 
current cycle, but 
the decline was less 
than 5 points. 


The school’s index 
fell more than 5 
points below the 
index for the 
present cycle. 


Consequences 


1 . Identified for 
Title I school 
improvement 

2. Required to 
develop a SIP 


1 . Identified for 
Title I school 
improvement 

2. Required to 
develop a SIP 


1 . Identified for 
Title I school 
improvement 

2. Required to 
develop a SIP 


1 . Identified for 
Title I school 
improvement 

2. Required to 
develop a SIP 

Teachers are placed 
on probation. 


Assistance 


Regional school 
support teams 
(SSTs) are notified 
of Title I schools 
identified for 
improvement. 

SSTs provide 
assistance with 
SIP and 
improvements. 


1 . Regional SST 
notified for 
Title I schools 

2. Assigned a DE 

3. Eligible for 
School 

Improvement 

Funds 


1 . Regional SST 
notified for 
Title I schools 

2. Assigned a DE 

3. Eligible for 
School 
Improvement 
Funds 


1. Regional SST 
notified for 
Title I schools 

2. Assigned a DE 

3. Eligible for 
School 
Improvement 
Funds 



*Note: Crisis category no longer exists under the new legislation. 
SIP = School Improvement Plan 
DE = Distinguished Educator 
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Appendix B: Comparison Tables for Accountability Policies—Maryland 



Maryland 





Title I School 
Improvement 


Reconstitution 

Eligible 


State Take-over 


Criteria 


1. Title I school 

2. Failure to meet or 
show substantial 
and sustained 
progress toward 
the standard of 
70% satisfactory 
on MSPAP and 
other elements 
incorporated in 
the SIP 


Identified by the state 
as a school that is 
below standard and 
fails to make progress 
or declines after 
developing and 
implementing a SIP 




Consequences 


Required to develop a 
SIP 




Operation of the 
school is handed over 
to an external agent. 


Assistance 


Varies by district 







SIP = School Improvement Plan. 
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New York 





School Improvement 


SURR 


Redesign 


Criteria 


Failing to meet the 90% 
standard and failure to 
make 20% 
improvement toward 
the 95 standard 


1 . Schools identified by 
the state as farthest 
from the standard. 

2. Schools identified as a 
“poor learning 
environment” 


Failure to Improve after 
SURR identification 


Consequences 


Required to develop a 
Comprehensive Action 
Plan 


1 . Districts are required to 
develop a corrective 
action plan that outlines 
their support of SURR 
schools. 

2. Schools must develop 
an improvement plan 
with the district. 

3. District is required to 
designate a special 
liaison to work with the 
SURR schools. 

4. Schools are identified 
as Corrective Action 
Schools following once 
the plans have been 
approved. 


1 . School must close 
and re-open under a 
new school plan. 


Support 


1 . Assistance for 
community school 
district to develop a 
local assistance 
plan (LAP) 

2. NYC school 
support teams assist 
with school-wide 
programs. 


1 . State provides 
registration review 
teams to assess needs 
of the school and 
resources needed. 

2. State provides a set of 
liaisons to help 
interpret regulations. 

3 . Continued monitoring 
of progress for up to 3 
years. 
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